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Introduction 


With support from the California Community College Chancellor’s Office, the MMAP Research 
Team, a team of The Research and Planning Group for California Community Colleges (RP 
Group) conducted a literature review on various assessment measures used for English as a 
Second Language (ESL) programs to explore effective assessment methods including 
guided/directed self-placement, writing samples/essays, various multiple measures, and the 
validitiy of International Baccalaureate (IB) program transcripts. We examined research on 
assessment methods for English Language Learner students (ELLs) and investigated whether 
tests such as the Test of English as a Foreign Language (TOEFL) exam, writing samples/essays, 
multiple measures questionaires, and guided self-placement are effective for course placement. 


The purpose of this report is to inform the AB 705 ESL Subcommittee of the AB 705 
Implementation Team the various assessment methods colleges use nationally for ELL students. 
The findings will also be useful to California Community Colleges that would like to further their 
understanding of the various methods used for the assessment of ELL students. With the 
passage of AB 705, high school transcripts are the primary source of assessment. However, a 
prior MMAP research brief for ESL (https://bit.ly/2BEmPD1) found that the vast majority (97%) 
of high school students did not take ESL courses at a community college, but instead enrolled in 
English for native speakers. Therefore, the ESL population for whom we have high school 
transcript data is limited. This is coupled with AB 705 limiting the use of assessment tests to 
only those approved by the Board of Governors, making alternative assessment measures 
extremely important to this population. 


This review of the literature begins with a summary of the main findings followed by a more 
detailed account of each article then concludes with a list of references. A file containing each 
full article is accessible here: https://bit.ly/2XEOOxF. 


Summary of Findings 


In mining the literature to incorporate into this report, the RP Group reviewed and selected 33 
scholarly articles addressing placement methods for ELL students. Though this literature review 
was not exhaustive, key themes include: 


Assessments Using Writing Samples and Essays 

Guided/Directed Self-Placement Assessment Methods 
Assessments Using Multiple Measures Questionaires 
Assessments Using Test of English as a Foreign Language (TOEFL) 
International Baccalaureate Program Assessments 


GB wm 
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Assessments Using Writing Samples and Essays 


This section provides a summary of the literature reviewed pertaining to writing samples or 
essays as a form of assessment for placement. 


Research in the area of ELL placement concurs that a writing sample or essay jointly reviewed 
with high school data is a better measure of success than a multiple-choice test (CCCC, 2001; 
Bunch, Endris, & Panayotova, 2011; Brunk-Chavez & Fredericksen, 2008; Matzen & Hoyt, 2004). 


Additionally, the Community College Committee on Second Language Writing (CCCC, 2001) 
made a statement regarding the placement decisions of ESL writers into writing courses: 


they should be based on students’ writing proficiency and rather than solely on the 
scores from standardized tests of general language or spoken language proficiency. 
Instead, scores from the direct assessment of students’ writing proficiency should be 
used, and multiple writing samples should be consulted whenever possible. They further 
advocate for the use of directed self-placement as a means of determining the most 
appropriate placement for second-language writers (CCCC, 2014, paragraph 16). 


When a writing sample is used, researchers assert the following considerations: 


e To increase the likelihood that test-takers are able to respond to a prompt that they 
fully understand, they should be able to either review the prompt ahead of time or 
choose from multiple prompts (Johnson & Riazi, 2017; Marini, Shaw, Young, & Hg, 
2008). 


e To help ensure test-takers are able to make revisions, an unlimited timeframe for a 
writing sample is advisable, as ELL students may process English more slowly than their 
peers who speak English as a first language (Crusan, 2002; Johnson & Riazi, 2017). 


e Rater consistency, reliability, and validity were found to be significant issues when the 
writing sample is human-scored. To mitigate this, research suggests that rater training 
and norming sessions are an important part of any writing assessment process (Johnson 
& Riazi, 2017; Karimi & Mehrdad, 2012). 


e Adecade of research suggests that the costs, resources, and time associated with using 
a writing sample as a placement measure appear to be a limitation for many colleges 
(Crusan, 2002; Bunch, Endris, & Panayotova, 2011; Karimi & Mehrdad, 2012). 


e Ongoing evaluation, bias testing, and validity testing of writing samples should also be 
part of the process (Johnson & Riazi, 2017; Karimi & Mehrdad, 2012). 


e Flexible re-testing policies should be in place (Bunch, Endris, & Panayotova, 2011). 
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Guided/Directed Self-Placement Assessment Methods 


Robust research exists around directed self-placement (DSP), better known in California as 
guided self-placement (GSP). Overall, there seems to be consensus that GSP has value ina 
placement process; it enables colleges to give students a voice in their placement and results 
in a valid placement that is helpful to students (Crusan, 2011; Ferris, Evans, & Kurzer, 2016; 
Inoue, 2009; LeBlanc & Painchaud, 1985; Royer & Gilles, 2003; Sinha, 2014). 


GSP most often involves providing students with ample examples, relevant context, course 
information, samples of work, and guidance so they may effectively place themselves into a 
level of course which they feel is best aligned with their learning thus far. GSP typically includes 
a self-assessment questionnaire along with an overview of the students’ course placement 
options. The “guided” portion of guided self-placement involves the inclusion of many factors 
that help students determine the best course for them, including past writing experience, 
confidence/efficacy with respect to writing, and descriptions of the courses themselves. 


One strength of GSP is the ability to adapt to the needs of students, programs, and colleges. 
Ultimately, GSP has been found to be a valid measure of assessment for ELL students and 
results in higher levels of success than other measures of assessment (Royer & Gilles, 2003; 
Ferris, 2016; NCTE, 2014; Ross, 2008; Inoue, 2009; Tompkins, 2003). 


Variations of GSP Models 


Colleges take different approaches to GSP, though the variations are not exhaustive, the RP 
Group selected the following to highlight: 


e Course information is provided to students through either an online survey or paper 
brochures about the possible classes that they can take. (Crusan, 2011). 


e Students complete surveys that ask questions about their past and present reading 
and writing habits and experiences. Questions cover a broad spectrum and may 
include items such as: “I like using computers to draft and revise writing,” “I turn in 
assignments on time, and no one has to remind me to get the work done,” 
“Generally, | don’t read when | don’t have to,” or “In high school, | wrote several 
essays per year.” (Crusan, 2011) 


e Students write an essay which is used to provide them with a real-time experience 
with which to base their writing self-evaluation (Bedore and Rossen-Knill, 2004). 


e Counseling or in-person guidance is provided to help students determine the 
appropriate class (Bedore & Rossen-Knill, 2004; Crusan, 2011). 


Review of Literature: Assessment Measures for ESL Students in College 
The RP Group | February 2019 | Page 5 


Recommendations when Developing a GSP Model 


Critical to the effectiveness of GSP, research suggests a number of recommendations for 
colleges interested in developing a guided self-placement model: 


Confirm the following key stakeholders are involved in the development of a GSP 
model: 1) faculty and classroom teachers, 2) students who need access to program 
information, 3) advisors who need to disseminate information and choices to 
students, and 4) administrators who make financial decisions (Blakesley, 2002). 


Provide information about what will be required of the student within each course, 
such as writing a research paper or multiple persuasive essays, as well as course 
expectations and learning outcomes (Sinha, 2014; Crusan, 2011). 


Equip students in advance with information about the GSP process, including sample 
questions and preparation materials as well as information about a challenge 
process (Blakesley, 2002; Bunch, Endris, & Panayotova, 2011; Crusan, 2011; Sinha, 
2014). 


Ensure information is readily available and consistent across the college. Information 
delivered in print, online, and verbally may be outdated, thus misdirecting students 
in their placement decision (Bedore & Rossen-Knill, 2004). 


Make publicly available videos or statements by students who have already 
completed the courses, offering a student perspective of what is required to 
successfully complete the course (Sinha, 2014). 


Ensure students are aware of and have access to support services, such as writing 
centers and tutoring (Sinha, 2014). 


Make GSP available to both international and residential second-language students 
(Crusan, 2011; NCTE, 2014). 


Develop online versions of GSP to assess international students off-campus using 
similar instruments, including actual writing samples and surveys about reading and 
writing habits and experiences (Crusan, 2011). 


Design self-assessments in a way that the content matches the curriculum of the 
course. In this way, what is expected of students is aligned with what they will learn 
in the course, which helps them make more of an informed decision about the 
course to enroll in (Ross, 2008). 


Offer clear and consistent information about the purpose of the GSP process and 
outcomes. This includes implications and consequences with respect to the college’s 
English and ESL sequences and the length of each sequence (Bedore & Rossen-Knill, 
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2004; Blakesley, 2002; Bunch, Endris, & Panayotova, 2011; Crusan, 2011; Sinha, 
2014). 


e Inform students of the advantages and disadvantages of each placement option in 
order to help them make informed decisions (NCTE, 2014). 


e Establish policies that allow flexibility regarding changing a student’s initial 
placement (Bunch, Endris, & Panayotova, 2011). 


Assessments Using Multiple Measures Questionnaires 


In this section, “multiple measures” refers to a pre-assessment questionnaire used to capture 
information about students’ backgrounds, skills, and abilities, among other things, as part of an 
assessment process. There is a somewhat limited amount of published literature regarding 
various types of multiple measures questionnaires used for placement of ELL students. 
However, the Multiple Measures Assessment Project (MMAP) (https://bit.ly/2EZRyxp) team has 
collected a number of multiple measures questions used by colleges and archived them 
(https://bit.ly/2EcBtTH). Further, MMAP has made one potential option for ESL multiple 
measures available in 10 different languages (https://bit.ly/2lij8IZ) that are being used and 
locally validated at several colleges statewide. 


In a California Community College study for academic ELL students (or those ELL students with a 
goal of degree or transfer), results showed that multiple measures and conversations with 
counselors were recommended to be a part of an informed placement process. The multiple 
measure assessment methods suggested in this study included K-12 assessment scores, 
classification as an English Learner (EL) or Redesignated Fluent English Proficient (R-FEP), Early 
Assessment Program (EAP) results, or a writing sample (Bunch, Endris & Panayotova, 2011). 


In another California Community College study for academic ELL students, Fagioli (2018) found 
four variables that correlated significantly with scores obtained on an ESL multiple measures 
survey: 


1) English self-rated proficiency (ranging from “proficient” to “low beginner,” with 
examples); 

2) The age at which the student started learning English (0-21 years old or 21+); 

3) The use of a translation sheet for the survey questions (yes or no); and 

4) Astudent’s experience reading books in English (never, yes) (Fagioli, 2018). 


Assessments Using TOEFL 


The Test of English as a Foreign Language (TOEFL) is a standardized assessment test used to 
measure the English language skills of non-native English speakers (citation). Four-year 
universities tend to use TOEFL as a form of assessment, and many community colleges have a 
minimum required score for admittance to international programs, but it is unclear how many 
use the TOEFL score as a form of course placement. Many colleges have asked the MMAP team 
about the validity of using TOEFL scores as a way to place ELL students. 
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Across numerous published studies and one California Community College study, the findings 
were similar, TOEFL scores were found to have a low correlation with academic achievement 
(Burgess & Greis, 1970; Ferris et al, 2016; Krausz, Schiff, Schiff, & Van Hise, 2005; Moglen, 2015; 
Newell, 2018; Ng, 2007; Wongtrirat, 2010). 


International Baccalaureate Program Assessments 


The International Baccalaureate (IB) program offers a uniform and accredited curriculum 
focused on international education at U.S. high schools or in other countries. Schools must be 
authorized to teach IB programs. Many California Community Colleges do not currently accept 
IB transcripts from domestic or international students. Literature pertaining to IB student 
success at four-year universities overwhelmingly shows IB graduates outperform non-IB 
students in a multitude of areas while at the university (citation). With this in mind, colleges 
should strongly consider the use of IB transcripts as a form of assessment. 


That said, IB students in general are just as likely as non-IB students to persist for over two 
years, earn high GPAs, and earn a college degree (citation). Further, IB students showed 
similar levels of metacognitive, cognitive, and college adjustment attitudes and behaviors 
across all areas assessed. 


Qualitative results found that IB students are more academically ready to successfully respond 
to the rigor and expectations of college courses than non-IB students. IB students reported 
feeling less intimidated by the heavy workload required in college courses, more comfortable 
having one final exam account for a large portion of their course grade, and more able to 
manage their time and workload efficiently (Bergeron et al., 2015; Coca et al, 2012; Conley, 
McGaughy, Davis-Molin, Farkas, & Fukuda, 2014; Halic, 2013). 


Detailed Summary of Articles 


This section provides a detailed summary of each article reviewed for readers who would like to 
dig a bit deeper into the findings. A reference to each article is included below for those readers 
who would like to download the full article. 


Articles Pertaining to Assessments Using Writing Samples and 
Essays 


Matzen and Hoyt (2004) argue that within the field of college composition, a timed essay exam 
is preferred if only one measure of placement into composition courses is used and if the only 
alternative is a multiple-choice test. Matzen and Hoyt elaborate, however, that timed essays 
have their own issues such as the reliability of essay scores, biases of essay prompts, and 
prescribed five-paragraph essay responses. Test anxiety and time limitations may also 
negatively affect the reliability of timed essay exams. The authors suggest that combining an 
essay with the student’s high school GPA can be more predictive of final grades in writing 
courses. 
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Within the study by Matzen and Hoyt (2004), 12 English teachers scored 431 timed essays 
written by incoming college freshmen in developmental English courses at a four-year 
university during the first week of classes. The freshmen students addressed one of two 
prompts and were allowed 30 minutes to write an essay. Matzen and Hoyt conclude that if only 
a single means of placement could be selected, and one had to choose between a reading test, 
an editing test, or a timed essay, the essay is preferable. Further, the timed essay shows less of 
a disparate impact on minority students compared to placement using a multiple-choice test. 


Marini et al. (2018) looked at the relationship between the scores of 4,500 students at 10 four- 
year universities received on the SAT essay and students first-semester grades in English and 
writing courses and first-semester GPA. The sample included over 4,500 students at 10 four- 
year universities. The SAT essay was developed to have a common prompt and SAT form- 
specific source texts. The same prompt is presented to students in every test, and students can 
access the prompt ahead of time. Marini et al. assert that a common prompt has a significant 
advantage for students compared to most essay tests because students are able to focus their 
preparation on developing important reading, analysis, and writing knowledge and skills instead 
of trying to guess what question will be asked on test day. Students can get right to work 
instead of spending considerable time trying to form an opinion on a topic they might not have 
spent much time thinking about. The authors found a positive trend in increasing GPA as scores 
on all parts of the essay—reading analysis, and writing—increased. 


Karimi and Mehrdad (2012) conducted a study of essays used for placement in a language 
department of Islamic Azad University of Hamedan where they reviewed 112 different essay 
tests. Karimi and Mehrdad provided suggestions to improve future essay tests. The main 
objective of the study was to suggest that teachers and other test-makers be cautious while 
designing tests and make themselves more knowledgeable about testing by studying the 
related literature. Authors further draw attention to the economy and administration in 
scoring the test, including cost per test. Essentially, authors convey, increased required 
personnel escalates the cost of the process. 


Brunk-Chavez and Fredericksen’s (2008) study looked at the reliability of the English 
ACCUPLACER multiple-choice test and holistically graded diagnostic essays at a four-year 
university. The results show that both are reliable predictors of success, but that two measures 
together make for an even more reliable predictor. The authors propose the use of an 
impromptu essay administered in 40 minutes of classroom time in the first week, which enables 
faculty to use the scores to quickly identify and support students in need of help. The college in 
which the study was conducted offers an English course with a corequisite lab for students who 
need additional support, and the one-unit lab is a mandatory class for students who perform 
poorly on the in-class diagnostic. The authors reference Matzen and Hoyt (2004), who indicate 
that when composition teachers score or rate timed essays, scores have been found to have a 
predictive relationship with final grades and to be more indicative of students’ writing 
abilities compared to multiple-choice test scores for the same students. 


Crusan (2002) performed an in-depth review of the literature pertaining to the assessment of 


ELL students at four-year universities. While most of the literature agrees that an indirect 
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assessment, such as multiple-choice tests, marginalize many ELL students, direct assessment of 
writing ability via an essay is a valid option, though it still poses challenges. One challenge 
Crusan raises is that with timed essays, second-language writers are unsure of the structures 
of the language and have difficulty producing fluent written discourse, especially in timed 
situations. Thus, timed writings may impose a disadvantage on many ELL students. Another 
issue is that timed writing tests are often given in artificial conditions under which students 
must compose based on an assigned topic and are unable to use reference materials, making 
it difficult to write “cold” on a topic they might never have seen before or know nothing 
about. 


Furthermore, Crusan (2002) notes that there is little revision in a timed setting. The author 
references the California Community College Committee on Second Language Writing, which 
remarks in its Statement on Second-Language Writing and Writers, 


Decisions regarding the placement of second-language writers into writing courses 
should be based on students’ writing proficiency and not based solely on the scores 
from standardized tests of general language proficiency or of spoken language 
proficiency. Instead, scores from the direct assessment of students’ writing proficiency 
should be used, and multiple writing samples should be consulted whenever possible. 
(2001) 


The Johnson and Riazi (2017) study intended to validate a locally created and rated writing test 
into college level English courses for English Foreign Language (EFL) students that resulted in 
robust findings for the institution, helping them improve their internal processes. Prior to the 
study, instructors who were involved in the development and evaluation of the assessment 
believed that the assessment overlapped the skills required in English classes and contributed 
to beneficial decision-making when placing students. However, the validation process made 
stakeholders aware that their perceptions were false; fortunately, it also provided a roadmap 
for how to improve the process. 


A significant proportion of test-takers reported difficulties understanding the instructions, 
prompts, and tasks, which led instructors to consider piloting future materials with current 
students and gathering insights via post-assessment questionnaires and/or focus groups. 
Presenting multiple prompts from which the test-takers may choose could increase the 
likelihood that applicants have the opportunity to respond to a prompt they clearly 
understand and allow more control over the topic and content of their writing (Johnson & Riazi, 
2017). 


Johnson and Riazi (2017) also found rater consistency to be an issue, and suggested that rater 
training and norming sessions be held in the future. The rubric was also problematic in that it 
was rather complicated and difficult to use. Additionally, although the test did not have a time 
limit, many test-takers reported perceiving time limits and feeling they were too short. 
Johnson and Riazi advise that prior to taking the test, all students should understand the 
purposes, possible outcomes, and processes involved in the placement testing system. The 
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study provides a viable validity framework that could be used by other colleges to implement 
ongoing evaluation and validity testing of essay assessments. 


Bunch, Endris, and Panayotova (2011) argue that even though almost half of California’s K-12 
students grew up in homes in which languages other than English were spoken, there is little 
awareness of this population within the community college system and little agreement 
regarding their characteristics and needs. To investigate, the authors conducted 51 telephone 
and in-person interviews with faculty and staff at 10 colleges representing different sizes, 
geographical areas, and student demographics. Bunch, Endris, and Panayotova reviewed 
college websites, policies, and practices related to testing and placement and examined the 
ways in which those policies and practices are communicated to students. 


Bunch, Endris, and Panayotova (2011) suggest that multiple measures coupled with 
conversations with counselors be a part of an informed placement process. Authors suggest the 
following should be included as a multiple measure include: K-12 assessment scores, 
classification as an ELL or Redesignated Fluent English Proficient (R-FEP) status, or Early 
Assessment Program (EAP) results. They also suggest the use of a writing sample in the 
placement process, though they note the expense and resources this requires. Bunch, Endris, 
and Panayotova further recommend liberal retesting policies for this population, as well as 
providing students with information about the testing process, sample test questions, test 
preparation materials, and information about a challenge process. The authors also argue that 
colleges should provide clear and consistent information about the purposes of the assessment 
and the outcomes as well as the implications and consequences of the English and ESL 
sequence and the length of each sequence. Lastly, the authors note that directed self- 
placement could be less expensive than using commercially developed placement tests and at 
one college in the sample, such placement for ESL students had proven to be as valid as the test 
formerly used by the college. 


Articles Pertaining to Guided/Directed Self-Placement 


Much of the literature pertaining to guided self-placement, as it is known in California, is 
referred to as directed self-placement (DSP) in much of the rest of the country (Ferris et al., 
2016). DSP typically includes a self-assessment questionnaire along with an overview of 
students’ course placement options. In some cases, students are asked to write a timed essay 
as well, but DSP is used mostly to provide students with a real-time experience with writing 
on which to base their own self-evaluation. According to Ferris et al., depending on the 
institution, students might meet with an advisor to discuss their choices, be allowed to enroll in 
the course they have selected after completing the directed self-placement instrument, or 
override an assessment provided to them via self-assessment. The “directed” portion of 
directed self-placement involves the inclusion of many factors that help the student determine 
the best courses for enrollment, including past writing experience and confidence/efficacy at 
writing, as well as descriptions of the courses themselves, expectations, learning outcomes, and 
experiences of past students. 


Review of Literature: Assessment Measures for ESL Students in College 
The RP Group | February 2019 | Page 11 


Ferris et al. (2016) emphasizes that while there are different approaches to DSP, there are also 
similarities. For example, information is provided about the possible classes students can take, 
and then students are asked to determine which is best aligned to their prior learning either 
through an online survey or paper brochures. Some colleges have students take a placement or 
diagnostic test and provide a recommendation based on performance, but students may 
override the test results. Other institutions provide counseling or other in-person guidance to 
students to help them determine the appropriate classes. Research shows that DSP’s ability to 
adapt to the needs of the students, programs, and colleges is its strength (Ferris et al.). 


As demonstrated in the following excerpt, it is important to acknowledge students’ potential 
anxiety and fears around using DSP or GSP for assessment (Ferris et al., 2016). Additionally, it is 
critical to acknowledge the risk that many instructors face: 


If students who are not fully aware (because of their differing cultural and educational 
experiences) of what language/writing proficiency entails, particularly in a demanding 
L2 academic environment, they might be more likely to aggressively place themselves so 
that they can make rapid progress through their degree requirements. Conversely, other 
students may lack confidence in their own abilities and place themselves lower than 
required. Not only can misplacement harm individual students themselves, but it can 
also make instructors’ jobs much more difficult (if they have students with widely 
varying abilities in the same writing class) and lead to broader programmatic problems 
(such as difficulties in administering end-of-course assessments and high failure rates) 


(p. 2). 


The authors conclude that DSP has value in the placement process and enables colleges to 
give students a voice in their own placement. 


As far back as 1985, LeBlanc and Painchaud discussed the use of self-assessment to place adult 
ELL learners. They argued that when adult students play a role in deciding their course 
enrollment, the experience yields an understanding of why they are entering a certain course 
and what they should expect from it. The National Council of Teachers of English (NCTE) 
published a position statement in 2014 for the Conference on College Composition and 
Communication. In Part Three: Guidelines for Writing Programs, First-Year Composition 
Placement, the organization asserts: 


..scores from the direct assessment of students’ writing proficiency should be used, and 
multiple writing samples should be consulted whenever possible. Writing programs 
should work toward making a wide variety of placement options available—including 
mainstreaming, basic writing, and second language writing as well as courses that 
systematically integrate native and nonnative speakers of English, such as cross-cultural 
composition courses. 


Placing residential second language students in appropriate college writing courses can 
be especially challenging because not all students self-identify as “ESL,” “multilingual” or 
“second language” students. Some students may welcome the opportunity to enrollina 
writing course designated for second language writers for the additional language 
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support while others may prefer to enroll in a mainstream first-year composition 
course. Due to these considerations, we advocate Directed Self-Placement as a means 
of determining the most appropriate placement for second language writers (for more 
information on Directed Self-Placement and a list of questions used, see Royer and 
Gilles, 1998). Writing programs should inform students of the advantages and 
disadvantages of each placement option so that students can make informed 
decisions, and should make this opportunity available to both international and 
residential second language students (NCTE, 2014). 


Royer and Gilles (2003) claim that directed self-placement promotes democracy in students’ 
education that results in agency, choice, and self-determination, factors that encourage 
students to feel invested in their education (Ferris et al. 2017, p. 4). Further, when students 
place themselves using DSP, they tend to have higher course success rates because they now 
feel the need to prove something. 


Ferris et al. (2017) studied directed self-placement of 1,067 ELL students at a public four-year 
university. As part of the study, students took a self-assessment survey as well as a locally 
developed placement exam. The results of both were compared to determine the level of 
alignment between the self-assessment and placement exam in placing students into a four- 
course writing program. 


The authors also conducted an extensive literature review of directed self-placement and noted 
particular interest in a book chapter in Royer & Gilles (2003) by Tompkins, who compares 
community college course grades to self-assessment scores and finds that DSP is just as 
effective as other forms of placement, with higher grades in a piloted GSP program than 
those placed from a timed test. 


Ferris et al. (2017) included a majority of international visa-holders (80-85%) as well as U.S. 
educated multilingual students. The ELL program at the college includes four levels, each one 
term long. Of the 1,067 students, most students’ (79%) self-placement scores were within one 
course level of their placement results while only about 20% were two or three course levels 
off. Over half (57%) did not place themselves at higher levels than the test. Ferris et al. further 
found that students at the bottom and top of the placement outcomes were more likely to 
over- or underrate themselves than the students in the middle. Authors conclude that program 
administrators and instructors should not discount ELL writers as unable to provide input 
regarding their placement outcomes. 


In TESOL Quarterly, Crusan (2011) opines, “The strength of a writing program often lies in its 
assessment techniques, so | strongly suggest that writing programs consider DSP as one option 
for placement of second language writers” (p. 2). Likewise, Crusan recognizes that there are 
factors that may negatively impact students’ placement using DSP, such as overconfidence in 
their skills and abilities, but when students are correctly informed about course expectations 
and their responsibilities as a learner in the course they select, misplacement is greatly 
reduced. Therefore, it is essential that programs tailor their DSP process to the needs of the 
college, program, and the students they serve. 
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Crusan (2011) offers examples of DSP across varied institutions. DSP may include face-to-face 
guidance from writing program administrators and teachers, PowerPoint presentations, 
websites, or brochures outlining the most important information for a student to know. 
Many colleges use surveys to ask questions about students’ past and present reading and 
writing experiences. Questions used include: “I like using computers to draft and revise 
writing,” “I turn in assignments on time and no one has to remind me to get the work done,” 
“Generally | don’t read when | don’t have to,” and “In high school, | wrote several essays per 
year.” (Crusan, 2011, p. 778). Some colleges also use online versions of DSP to assess 
international students away from the campus using similar instruments. According to Crusan, 
these include actual writing samples, demographic questionnaires, and surveys about reading 
and writing habits and experiences. 


Crusan (2011) acknowledges that some TESOL educators may still be weary of DSP and fear that 
English language learners are not equipped to assess themselves correctly, but with the 
addition of directing or guiding students through the process, DSP can be a valuable asset to an 
assessment process for second language writers. She further argues that DSP sends a message 
to students that they are important and that colleges value what they have to say, affording 
them agency in the placement decision. Crusan concludes that while there may be no right 
answer to how best to place students, DSP deserves more than a passing glance. 


Inoue (2009) performed an interesting assessment of a DSP model used for the English writing 
program at California State University (CSU), Fresno. While DSP was not used with an ELL 
population, there were interesting findings regarding DSP for course placement purposes. 
Inoue posits that DSP contributes to increased agency and responsibility among students over 
their educational path. 


In order to assess how well students were self-assessing into writing courses, the faculty in 
Inoue’s (2009) study looked at entry and exit course surveys, portfolio ratings by teachers and 
peers, and course grades. Upon review of these measures, the faculty felt that all students had 
“adequate quality” or better on their final portfolio and found that satisfaction with the 
course improved from start to finish. Additionally, looking at retention rates, faculty found 
their program to have a positive effect on student retention, especially for those students 
who were designated as needing remediation by the university. Inoue argues that when 
students do well in this writing program that provides students with agency and choice via the 
DSP, they stay in the university longer. 


Ross (2008) conducted a meta-analysis of self-assessment (SA) as a form of second-language 
assessment. The author notes that SA has been widely used and researched, though this 
research has resulted in varied findings. In many articles, SA is considered a viable alternative to 
formal assessment tests, but in others, SA is found to lack validity. The article summarizes the 
research literature on 60 correlations reported in the second-language testing literature 
pertaining to listening, reading, and writing tests. 


Ross (2008) points to the need to design self-assessments in a way that the content of the self- 
assessment items match the criterion skills, thus designing the self-assessment of language 
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learning achievement according to specific curricular content. If the content validity 
requirement is met, the author argues that there is clear potential for predictive accuracy of 
criterion skills based on self-assessment measures. However, when there is a mismatch 
between the content of the self-assessment items and criterion skills, the self-assessment items 
contributed very little to achievement test variance. Bunch, Endris, and Panayotova (2011) 
recommend that directed self-placement could be less expensive than using commercially 
developed placement tests and note that at one college in the sample, such placement for 
ESL has proven to be as valid as the test formerly used by the college. They also suggest liberal 
retesting policies for this population as well as providing students with information about the 
testing process, sample test questions, test preparation materials, and information about a 
challenge process. 


Extending the DSP approach, Bedore and Rossen-Knill (2004) expound upon the Informed Self 
Placement (ISP) model employed at the University of Rochester (UR). While UR’s ISP model is 
for placement into English courses (transfer-level or transfer-level with a support lab attached), 
and the institution is rather small with 950 new students each fall, it has found success by 
engaging faculty in ISP. The model requires that students write an essay that is then read by a 
faculty member, and a placement is given. The student can request to speak to a faculty 
member about their placement and discuss the work together to make an informed choice. The 
student can ultimately override the faculty member’s placement decision, resulting in an 
informed self-placement. Should a student seek additional consultation regarding their 
placement, Bedore and Rossen-Knill include guidelines for faculty. 


While this type of DSP is labor-intensive and may only work at a small institution, the study 
provides some important findings. Bedore and Rossen-Knill (2004) note that they set out to 
determine whether students could responsibly determine their writing placement levels, 
recognizing that students might lack necessary information to make this choice or could make a 
mistake in the process. Their assessment results showed that when they provided a great deal 
of information to students, students made the choice and students judged that choice to be 
appropriate to their knowledge level once enrolled in the course. Lastly, they learned that the 
choices students make are only as good as the information provided to them. Therefore, 
colleges must ensure that information is readily available and consistent across the college. 
The authors found that outdated information in print, online, and from people who are 
unaware of recent changes produce misinformation that must be addressed. 


Blakesley (2002) acknowledges the challenges that faculty and staff at most universities face 
when implementing a DSP model. The author asserts that universities must relinquish power 
and give it to the students. Blakesley, Harvey and Reynolds (2003) identify four major groups of 
stakeholders they believe should be included in the process and convinced of the importance 
and value of DSP in order for it to function well: 1) students who need access to program 
information, 2) student advisors who need to disseminate information and choices to 
students, 3) administrators who make financial decisions, and 4) teachers in the classroom. 


Sinha (2014) studied DSP within the California State University system. The author observes 
that DSP allows students an educational choice of two or more courses, usually the writing 
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course or the writing course as a stretch (two-term) course, and helps them make that decision 
based on their own assessment of their writing abilities. Thus, it is important that all students 
have access to support services such as writing help. Moreover, students should have 
information regarding course expectations, learning outcomes, and assignments, such as 
writing a research paper or multiple persuasive essays. Providing examples of each different 
type of assignment as well as statements or information provided by students who already 
completed the course is another option. 


Sinha (2014) also describes advantages and disadvantages of using DSP. In terms of advantages, 
Sinha posits that DSP gives students a fair chance to prove their abilities and offers students the 
agency to choose their own courses. Moreover, DSP is locally situated and allows colleges the 
opportunity to ensure placement is transparent, meaningful, and context-specific. With 
respect to disadvantages, Sinha notes that some students will make wrong decisions for 
themselves; therefore, teachers and administrators need to participate in the decision-making 
process to ensure students have as much information as possible to facilitate effective course 
selection. In addition, DSP requires more planning and preparedness than standard assessment 
tests. 


Articles Pertaining to Multiple Measures Questionaires 


In 2018, Fagioli et al. developed a survey at Irvine Valley College to use as potential multiple 
measures for ELL students without high school transcript data. Researchers recruited four other 
community colleges to also administer the survey to their potential ELL student populations for 
a total sample of 4,099 students. They obtained the survey results as well as course enrollment 
and success data. Out of 15 questions on the survey, they found four to be correlated with ESL 
placement level (1-6 levels below transfer-level): 


1) English self-rated proficiency (ranging from “proficient” to “low beginner,” with 
examples); 

1) The age at which the student started learning English (0-21 years old or 21+); 

2) The use of a translation sheet for the survey questions (yes or no); and 

3) Astudent’s experience reading books in English (never, yes) (Fagioli, 2018). 


Students who placed one level below transfer-level had a self-rated proficiency of low 
advanced, advanced or proficient, had learned English prior to age 21 and did not use a 
translation sheet. 


Articles Pertaining to the use of the TOEFL Exam for Assessment 


As stated earlier, four-year universities tend to use TOEFL as a form of assessment, and many 
community colleges have a minimum required score for admittance to international programs, 
but it is unclear how many use the TOEFL score as a form of course placement. The literature 
reviewed suggests the TOEFL scores have limited value for placement. 
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Ferris et al. (2016) found in a national investigation of English as second language (ELL) writing 
placement procedures at private and public U.S. colleges that 26% of colleges responding to the 
survey used only the Test of English as a Foreign Language (TOEFL) scores for placement. The 
authors reported that a “dominant majority” of programs studied used some version of the 
TOEFL for placement purposes. However, the authors found TOEFL scores overall, as well as 
specific writing and speaking scores, to have small correlations with standardized test scores, 
suggesting they have limited predictive value for placement purposes. 


Moglen (2015) found moderate correlations (0.52) between scores on TOEFL and a local 
placement exam. However, the author states that in spite of this moderate correlation, there is 
no evidence that TOEFL scores predict academic achievement. The Education Testing Services 
(ETS), which owns and administers TOEFL, states on its website that assessment decisions 
should not be based solely on the TOEFL score; rather, the score should be used in conjunction 
with other criteria. 


Research in the field generally corroborates this stance. For example, Wongtrirat (2010) 
conducted a meta-analysis of the predictive qualities of TOEFL, in particular its ability to predict 
academic achievement, defined as “GPA, numbers of courses completed, or both” (p. 14). The 
results of the analysis showed that TOEFL scores had a low positive correlation with academic 
achievement, including findings from Ng (2007), who looked at 433 international students, as 
well as from Krausz, Schiff, Schiff, and Van Hise (2005) and Zhang (1996).Rarely was a high 
correlation reported, and when it was, as in the study by Burgess and Greis (1970), the results 
were not generalizable because of the small sample size. Research based on one California 
Community College (Newell, 2018) produced similar findings when correlating the local ESL 
placement test with TOEFL scores as well as correlating TOEFL scores with ESL course success. 


Articles Pertaining to the Use of International Baccalaureate 
Program Transcript Data for Assessment 


As stated earlier, the International Baccalaureate (IB) offers international education within the 
U.S. and worldwide. Schools must be authorized to teach IB programs. Many California 
Community Colleges do not currently accept IB transcripts from students within the U.S. or 
internationally. This review of the literature indicates that colleges should strongly consider the 
use of IB transcripts as a form of assessment as the literature pertaining to IB student success at 
four-year universities overwhelmingly showed IB graduates outperformed non-IB students in a 
multitude of areas while at the university. 


Conley, McGaughy, Davis-Molin, Farkas, and Fukuda (2014) explored the impact of high school 
International Baccalaureate Diploma programs on students’ preparation for college by looking 
at both academic and non-academic factors among University of Oregon students, some of 
whom had completed high school IB programs and some who had not. The authors examined 
students’ academic, social, and emotional adjustment and the factors that helped transition 
both groups from high school to college. The results suggest that IB students in general are just 
as likely to persist for over two years, earn high GPAs, and earn a college degree compared to 


Review of Literature: Assessment Measures for ESL Students in College 
The RP Group | February 2019 | Page 17 


non-IB students. Further, IB students showed similar levels of metacognitive, cognitive, and 
college adjustment attitudes and behaviors across all areas assessed. 


The qualitative results found that IB students are more academically ready for the rigor and 
expectations of college courses than non-IB students. IB students reported feeling less 
intimidated by the heavy workload required in college courses, more comfortable having one 
final exam account for a large portion of their course grades, and better able to manage their 
time and workload efficiently. Overall, this study found that IB students were better prepared 
for college in terms of both academic and non-academic factors, and no differences were 
detected in the GPAs between groups. Additionally, IB students who completed four years of 
high school were more likely to complete college than non-IB students. 


Halic (2013) conducted a longitudinal study of over 11,000 IB graduates over six years, 
examining postsecondary enrollment, retention, and graduation. The author used National 
Student Clearinghouse data for graduates who sat for the IB exam in 2005, which resulted in a 
match of 9,654 students included in the analysis. Of all graduates of IB programs in 2005, 92% 
enrolled at a postsecondary institution, with 10% (872) enrolling in California. Almost three- 
quarters (74%) graduated within four years, compared to the national average of 38%. Overall, 
the longitudinal study found higher persistence, retention, and graduation rates for students 
who graduated from an IB program when compared to national rates. 


Coca et al. (2012) examined the impact of Chicago’s neighborhood IB programs on 
postsecondary outcomes of graduates between 2003 and 2007. The authors used propensity 
matching and student interviews to investigate student experiences. Overall, they found that IB 
students were 40% more likely to attend a postsecondary institution and persisted to 
graduation at a higher rate than non-IB students. When in college, IB students reported 
feeling prepared to succeed and excel in coursework. Bergeron et al. (2015) also found similar 
results looking at IB graduates between 2008 and 2014. 
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